Head-mounted display device and control method for the head-mounted display device

ABSTRACT

A head-mounted display device includes: an image display unit including an image-light generating unit that generates image light representing an image and emits the image light and a light guide unit that guides the emitted image light to the eye of a user, the image display unit being for causing the user to visually recognize a virtual image; and a control unit that includes an operation surface, is connected to the image display unit, and controls image display by the image display unit. When it is assumed that the user shifts the user&#39;s attention from the virtual image, the control unit adjusts the luminance of the image-light generating unit or adjusts the image light generated by the image-light generating unit to reduce the visibility of the virtual image.

This is a Continuation of application Ser. No. 13/419,010 filed Mar. 13,2012, which claims the benefit of Japanese Patent Application No.2011-066373 filed Mar. 24, 2011 and Japanese Patent Application No.2011-066393 filed Mar. 24, 2011. The disclosure of the priorapplications is hereby incorporated by reference herein in its entirety.

BACKGROUND

1. Technical Field

The present invention relates to a head-mounted display device, which isa display device mounted on the head, and a control method for thehead-mounted display device.

2. Related Art

There is known a head-mounted display device (a head mounted display,HMD), which is a display device mounted on the head. For example, thehead-mounted display device generates image light representing an imageusing a liquid crystal display and a light source and guides thegenerated image light to the eyes of a user using a projection opticalsystem and a light guide plate to thereby cause the user to recognize avirtual image.

In the head mounted display explained above, when the user operates acontrol unit of the head mounted display for the purpose of, forexample, playback, stop, and fast forward of a video, it is likely thata virtual image displayed before the eyes of the user blocks the visualfield of the user and hinders the operation. In order to solve such aproblem, in the past, there is known a technique for picking up an imageof a range including a display region with a camera, analyzing thepicked-up image to detect the hands of the user, and reducing thevisibility of an image displayed in a region formed between one hand andthe other hand of the detected hands to improve the visibility of anexternal scene (e.g., JP-A-2010-211408).

With the head mounted display explained above, the user can enjoy images(videos) and music anywhere while wearing the head mounted display like,for example, eyeglasses. On the other hand, in a worn state of the headmounted display, since an image blocking an external scene is alwaysdisplayed before the eyes of the user, the user could feelinconvenience. In order to solve such a problem, in the past, there isknown a technique for detecting that a user is walking, automaticallystopping image display of the head mounted display, and improving thevisibility of an external scene (e.g., JP-A-09-211382).

Besides, JP-A-09-211382, JP-A-2008-116704, JP-A-2003-51993,JP-A-2007-134785, and JP-A-2004-96224 are examples of related art.

However, in the technique in the past for detecting the hands of a user,for example, it is likely that a malfunction due to misdetection occursin a use in a public space. In the technique in the past for detectingthat a user is walking, inconvenience for a walking user is reduced.However, in the past, there is a demand for also reducing, even when theuser is not walking, inconvenience that the user feels, for example,when the user turns the user's face away from an image displayed on thehead mounted display to shift the user's attention from the image.

SUMMARY

An advantage of some aspect of the invention is to provide, in ahead-mounted display device, a technique for improving the visibility ofan external scene when it is assumed that a user shifts the user'sattention from a virtual image.

Application Example 1

This application example of the invention is directed to a head-mounteddisplay device including: an image display unit including an image-lightgenerating unit that generates image light representing an image andemits the image light and a light guide unit that guides the emittedimage light to the eye of a user, the image display unit being forcausing the user to visually recognize a virtual image; and a controlunit that is connected to the image display unit and controls imagedisplay by the image display unit. When it is assumed that the usershifts the user's attention from the virtual image, the control unitadjusts the luminance of the image-light generating unit or adjusts theimage light generated by the image-light generating unit to reduce thevisibility of the virtual image.

With such a configuration, when it is assumed that the user shifts theuser's attention from the virtual image, the control unit adjusts theluminance of the image-light generating unit or adjusts the image lightgenerated by the image-light generating unit to reduce the visibility ofthe virtual image. Therefore, in the head-mounted display device, whenit is assumed that the user shifts the user's attention from the virtualimage, it is possible to improve the visibility of an external scene.

Application Example 2

This application example of the invention is directed to thehead-mounted display device of Application Example 1, which furtherincludes a direction determining unit that determines whether the userfaces an operation surface of the control unit. The control unit furtherincludes the operation surface. When it is determined that the userfaces the operation surface, the control unit adjusts the luminance ofthe image-light generating unit or adjusts the image light generated bythe image-light generating unit to reduce the visibility of the virtualimage.

With such a configuration, the direction determining unit determineswhether the user faces the operation surface of the control unit. Whenit is determined that the user faces the operation surface, the controlunit adjusts the luminance of the image-light generating unit or adjuststhe image light generated by the image-light generating unit to reducethe visibility of the virtual image. Therefore, in the head-mounteddisplay device, it is possible to detect, with a configuration differentfrom that in the past, that the user attempts to operate the controlunit of the head-mounted display device. It is possible to improve thevisibility of an external scene in the image display unit.

Application Example 3

This application example of the invention is directed to thehead-mounted display device of Application Example 2, which furtherincludes: a head-side light emitting unit that is arranged in the imagedisplay unit and emits invisible light; and a control-unit-side lightreceiving unit that is arranged on the operation surface and receivesthe emitted invisible light. The direction determining unit determineswhether the user faces the operation surface of the control unit bydetermining, using an output signal of the control-unit-side lightreceiving unit, whether the operation surface and the image display unitare opposed to each other.

With such a configuration, the head-side light emitting unit that emitsvisible light is arranged in the image display unit and thecontrol-unit-side light receiving unit that receives the emittedinvisible light is arranged on the operation surface. Therefore, thedirection determining unit can determine whether the user faces theoperation surface of the control unit by determining, using an outputsignal of the control-unit-side light receiving unit, whether theoperation surface and the image display unit are opposed to each other.

Application Example 4

This application example of the invention is directed to thehead-mounted display device of Application Example 3, which furtherincludes: a control-unit-side light emitting unit that is arranged inthe operation surface and emits invisible light; and a head-side lightreceiving unit that is arranged in the image display unit and receivesthe emitting invisible light. The control unit causes the head-sidelight emitting unit and the control-unit-side light emitting unit toalternately emit the invisible light. The direction determining unitdetermines whether the user faces the operation surface of the controlunit by determining, using an output signal of the control-unit-sidelight receiving unit and an output signal of the head-side lightreceiving unit, whether the operation surface and the image display unitare opposed to each other.

With such a configuration, the control-unit-side light emitting unitthat emits invisible light is further arranged on the operation surfaceand the head-side light receiving unit that receives the emittedinvisible light is further arranged in the image display unit. Thecontrol unit causes the head-side light emitting unit and thecontrol-unit-side light emitting unit to alternately emit the invisiblelight. The direction determining unit determines whether the user facesthe operation surface of the control unit by determining, using anoutput signal of the control-unit-side light receiving unit and anoutput signal of the head-side light receiving unit, whether theoperation surface and the image display unit are opposed to each other.Therefore, in the head-mounted display device including the directiondetermining unit, it is possible to improve accuracy of thedetermination by the direction determining unit.

Application Example 5

This application example of the invention is directed to thehead-mounted display device of Application Example 3 or 4, which furtherincludes a contact detecting unit that is arranged in the control unitand for detecting contact with the control unit. When the contact withthe control unit is detected, the control unit causes the head-sidelight emitting unit to emit the invisible light and causes thecontrol-unit-side light emitting unit to emit the invisible light.

With such a configuration, the contact detecting unit for detectingcontact with the control unit is arranged in the control unit.Therefore, when the contact with the control unit is detected, thecontrol unit can cause the head-side light emitting unit to emit theinvisible light and cause the control-unit-side light emitting unit toemit the invisible light.

Application Example 6

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 3 to 5,wherein the invisible light emitted by the head-side light emitting unitincludes identification information for identifying the image displayunit according to pulse modulation of the invisible light. The controlunit includes a storing unit that stores authentication information forauthenticating the image display unit connected to the control unit. Thedirection determining unit further includes an authenticating unit thatauthenticates the image display unit by acquiring the identificationinformation from an output signal of the control-unit-side lightreceiving unit and performing a search as to whether the acquiredidentification information is included in the authenticationinformation.

With such a configuration, the invisible light emitted by the head-sidelight emitting unit includes the identification information foridentifying the image display unit. Therefore, the authenticating unitcan authenticate the image display unit by acquiring the identificationinformation from an output signal of the control-unit-side lightreceiving unit and performing a search as to whether the acquiredidentification information is included in the authenticationinformation. As a result, it is possible to reduce misrecognition.

Application Example 7

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 2 to 6, whichfurther includes: an acceleration-information detecting unit that isarranged in the image display unit and for detecting accelerationinformation of the image display unit; and anangular-velocity-information detecting unit that is arranged in theimage display unit and for detecting angular velocity information of theimage display unit. The direction determining unit determines whetherthe user faces the operation surface of the control unit by determining,using the detected acceleration information and the detected angularvelocity information, whether the operation surface and the imagedisplay unit are opposed to each other.

With such a configuration, the acceleration-information detecting unitfor detecting acceleration information of the image display unit and theangular-velocity-information detecting unit for detecting angularvelocity information of the image display unit are arranged in the imagedisplay unit. Therefore, the direction determining unit can determine,using the detected acceleration information and the detected angularvelocity information, whether the user faces the operation surface ofthe control unit.

Application Example 8

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 2 to 7,wherein the image-light generating unit includes: a display device thatgenerates the image; and a light source that emits image lightrepresenting the generated image. When it is determined that the userfaces the operation surface of the control unit, the control unitadjusts the luminance of the image-light generating unit by turning offor reducing illumination light of the light source.

With such a configuration, when it is determined that the user faces theoperation surface of the control unit, the control unit adjusts theluminance of the image-light generating unit by turning off or reducingillumination light of the light source. Therefore, it is possible toimprove the visibility of an external scene in the image display unit.

Application Example 9

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 2 to 7,wherein the image-light generating unit includes: a display device thatgenerates the image; and a light source including plural light emittingmembers for emitting image light representing the generated image. Whenit is determined that the user faces the operation surface of thecontrol unit, the control unit adjusts the luminance of the image-lightgenerating unit by turning off or reducing illumination light of atleast a part of the plural light emitting members.

With such a configuration, when it is determined that the user faces theoperation surface of the control unit, the control unit adjusts theluminance of the image-light generating unit by turning off or reducingillumination light of at least a part of the plural light emittingmembers. Therefore, it is possible to improve the visibility of anexternal scene in a part of the image display unit.

Application Example 10

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 2 to 7,wherein the control unit transmits image data to the image-lightgenerating unit. When it is determined that the user faces the operationsurface of the control unit, the control unit adjusts the image lightgenerated by the image-light generating unit by replacing at least apart of the image data to be transmitted with dummy data indicatingblack.

With such a configuration, when it is determined that the user faces theoperation surface of the control unit, the control unit adjusts theimage light generated by the image-light generating unit by replacing atleast a part of the image data transmitted to the image-light generatingunit with dummy data indicating black. Therefore, it is possible toimprove the visibility of an external scene in the image display unit.

Application Example 11

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 2 to 7,wherein the image-light generating unit includes: a display device thatgenerates the image; and a light source that emits image lightrepresenting the generated image. When it is determined that the userfaces the operation surface of the control unit, the control unitadjusts the image light generated by the image-light generating unit byreducing a liquid crystal aperture ratio of at least a part of thedisplay device.

With such a configuration, when it is determined that the user faces theoperation surface of the control unit, the control unit adjusts theimage light generated by the image-light generating unit by reducing aliquid crystal aperture ratio of at least a part of the display device.Therefore, it is possible to improve the visibility of an external scenein the image display unit.

Application Example 12

This application example of the invention is directed to thehead-mounted display device of Application Example 1, which furtherincludes a detecting unit that acquires change information indicating achange in the direction of the image display unit and detects, using thechange information, the movement of the head exceeding a fixed amount ofthe user wearing the image display unit. When the movement of the headexceeding the fixed amount is detected, the control unit adjusts theluminance of the image-light generating unit or adjusts the image lightgenerated by the image-light generating unit to reduce the visibility ofthe virtual image.

With such a configuration, the detecting unit acquires changeinformation indicating a change in the direction of the display unit anddetects, using the change information, the movement of the headexceeding the fixed amount of the user wearing the image display unit.When the movement of the head exceeding the fixed amount is detected,the control unit adjusts the luminance of the image-light generatingunit or adjusts the image light generated by the image-light generatingunit to reduce the visibility of the virtual image. Therefore, in thehead-mounted display device, it is possible to detect the movement ofthe head exceeding the fixed amount of the user wearing the imagedisplay unit and improve the visibility of an external scene.

Application Example 13

This application example of the invention is directed to thehead-mounted display device of Application Example 12, wherein thedetecting unit sets, according to occurrence of a trigger set inadvance, an initial position, which is a position serving as a referencein detecting the movement of the image display unit, acquires the changeinformation with respect to the initial position, and detects, using thechange information, the movement of the head exceeding the fixed amount.

With such a configuration, the detecting unit sets an initial position,which is a position serving as a reference in detecting the movement ofthe image display unit, and acquires the change information with respectto the initial position. Therefore, it is possible to detect, using thechange information, the movement of the head exceeding the fixed amountof the user wearing the image display unit.

Application Example 14

This application example of the invention is directed to thehead-mounted display device of Application Example 13, wherein thedetecting unit specifies the initial position and the change informationaccording to a combination of the angle of the head corresponding to themovement of the head in the vertical direction of the user wearing theimage display unit and the direction of the face of the usercorresponding to the movement of the face in the horizontal direction.

With such a configuration, the detecting unit can specify the initialposition and the change information according to a combination of theangle of the head corresponding to the movement of the head in thevertical direction of the user wearing the image display unit and thedirection of the face of the user corresponding to the movement of theface in the horizontal direction.

Application Example 15

This application example of the invention is directed to thehead-mounted display device of Application Example 12 or 13, wherein theimage display unit further includes an angular-velocity detecting unitthat detects the angular velocity of the image display unit. Thedetecting unit acquires the angular velocity detected as the changeinformation. When an angle calculated from the angular velocity exceedsa first threshold set in advance and the angular velocity exceeds asecond threshold set in advance, the detecting unit determines that themovement of the head exceeding the fixed amount is detected.

With such a configuration, when an angle calculated from an angularvelocity detected by the angular-velocity detecting unit exceeds thefirst threshold set in advance and the angular velocity detected by theangular-velocity detecting unit exceeds the second threshold set inadvance, the detecting unit determines that the movement of the headexceeding the fixed amount is detected. Therefore, it is possible toneglect the small movement of the user by appropriately setting thefirst threshold and neglect the slow movement of the user byappropriately setting the second threshold.

Application Example 16

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 12 to 15,wherein the image display unit further includes an accelerationdetecting unit that detects the acceleration of the image display unit.The detecting unit further acquires the detected acceleration as thechange information and determines, using the acceleration and the tiltof the image display unit calculated from the acceleration, whether themovement of the head exceeding the fixed amount is detected.

With such a configuration, the detecting unit further determines,further using the tilt of the image display unit calculated from theacceleration detected by the acceleration detecting unit, whether themovement of the head exceeding the fixed amount is detected. Therefore,it is possible to improve accuracy of the determination in the detectingunit.

Application Example 17

This application example of the invention is directed to thehead-mounted display device of Application Example 12 or 13, wherein theimage display unit further includes: a terrestrial-magnetism detectingunit that detects the direction of the image display unit usingterrestrial magnetism; and an acceleration detecting unit that detectsthe acceleration of the image display unit. The detecting unit acquiresthe detected direction and the detected acceleration as the changeinformation. When an angle calculated from the direction exceeds a firstthreshold set in advance and the acceleration exceeds a second thresholdset in advance, the detecting unit determines that the movement of thehead exceeding the fixed amount is detected.

With such a configuration, when an angle calculated from the directiondetected by the terrestrial-magnetism detecting unit exceeds the firstthreshold set in advance and the acceleration detected by theterrestrial-magnetism detecting unit exceeds the second threshold set inadvance, the detecting unit determines that the movement of the headexceeding the fixed amount is detected. Therefore, it is possible toobtain effects same as those obtained in Application Example 15, withthe configuration in which the terrestrial-magnetism detecting unit andthe acceleration detecting unit are provided in place of theangular-velocity detecting unit in Application Example 15.

Application Example 18

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 12 to 17,wherein the trigger set in advance is at least any one of power-on ofthe head-mounted display device, detection of startup of a predeterminedapplication, and detection of pressing of a predetermined button.

With such a configuration, it is possible to set, according to at leastany one of power-on of the head-mounted display device, detection ofstartup of a predetermined application, and detection of pressing of apredetermined button, an initial position that should be a reference indetecting the movement of the image display unit.

Application Example 19

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 12 to 18,wherein the image-light generating unit includes: a display device thatgenerates the image; and a light source that emits image lightrepresenting the generated image. When the movement of the headexceeding the fixed amount is detected, the control unit adjusts theluminance of the image-light generating unit by turning off or reducingillumination light of the light source.

With such a configuration, when the movement of the head exceeding thefixed amount is detected, the control unit adjusts the luminance of theimage-light generating unit by turning off or reducing illuminationlight of the light source. Therefore, it is possible to improve thevisibility of an external scene in the image display unit.

Application Example 20

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 12 to 18,wherein when the movement of the head exceeding the fixed amount isdetected, the control unit adjusts the image light generated by theimage-light generating unit by temporarily stopping the generation ofimage light by the image-light generating unit.

With such a configuration, when the movement of the head exceeding thefixed amount is detected, the control unit adjusts the image lightgenerated by the image-light generating unit by temporarily stopping thegeneration of image light by the image-light generating unit. Therefore,it is possible to improve the visibility of an external scene in theimage display unit.

Application Example 21

This application example of the invention is directed to thehead-mounted display device of any of Application Examples 12 to 20,wherein the image display unit includes an image pick-up unit that picksup an image of the eyeball of the user. The detecting unit furtheracquires a line-of-sight movement amount indicating a movement amountwith respect to the center position of the iris of the user by analyzingthe picked-up image of the eyeball. When the line-of-sight movementamount exceeds a third threshold set in advance, the control unitfurther adjusts the luminance of the image-light generating unit oradjusts the image light generated by the image-light generating unit toreduce the visibility of the virtual image.

With such a configuration, the detecting unit acquires a line-of-sightmovement amount indicating a movement amount with respect to the centerposition of the iris of the user by analyzing the picked-up image of theeyeball. When the line-of-sight movement amount exceeds the thirdthreshold set in advance, the control unit further adjusts the luminanceof the image-light generating unit or adjusts the image light generatedby the image-light generating unit to reduce the visibility of thevirtual image. Therefore, in addition to the effect that it is possibleto improve the visibility of an external scene, it is possible to detectthat the user shifts the user's line of sight and improve the visibilityof an external scene.

The invention can be realized in various forms. For example, theinvention can be implemented in forms of a head-mounted display deviceand a control method for the head-mounted display device, a head-mounteddisplay system, a computer program for realizing functions of theapparatus, the method, or the system, and a recording medium having thecomputer program stored therein.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is an explanatory diagram showing the configuration of theexterior of a head-mounted display device in a first embodiment of theinvention.

FIG. 2 is a functional block diagram showing the configuration of a headmounted display.

FIG. 3 is an explanatory diagram showing a state in which image light isemitted by an image-light generating unit.

FIG. 4 is an explanatory diagram showing an example of a virtual imagerecognized by a user.

FIG. 5 is a flowchart for explaining a procedure of user-sidesee-through processing for the head mounted display.

FIG. 6 is an explanatory diagram showing a state of the image-lightgenerating unit in step S106 in FIG. 5.

FIGS. 7A and 7B are explanatory diagrams showing states in which theuser-side see-through processing (FIG. 5) is executed.

FIG. 8 is an explanatory diagram showing another processing form in stepS106 of the user-side see-through processing (FIG. 5).

FIG. 9 is an explanatory diagram showing the configuration of theexterior of a head mounted display in a second embodiment.

FIG. 10 is a functional block diagram showing the configuration of thehead mounted display in the second embodiment.

FIG. 11 is a flowchart for explaining a procedure of user-sidesee-through processing for the head mounted display in the secondembodiment.

FIG. 12 is an explanatory diagram showing the configuration of theexterior of a head-mounted display in a third embodiment.

FIG. 13 is a functional block diagram showing the configuration of ahead mounted display in the third embodiment.

FIGS. 14A and 14B are explanatory diagrams for explaining movementdetection processing.

FIG. 15 is a flowchart for explaining a procedure of the movementdetection processing.

FIG. 16 is an explanatory diagram showing a state of an image-lightgenerating unit in step S414 in FIG. 15.

FIGS. 17A and 17B are explanatory diagrams showing states in which themovement detection processing is executed.

FIG. 18 is a functional block diagram showing the configuration of ahead mounted display in a fourth embodiment.

FIG. 19 is a flowchart for explaining a procedure of line-of-sightdetection processing for the head mounted display in the fourthembodiment.

FIGS. 20A to 20C are explanatory diagrams concerning a method ofanalyzing an eyeball image.

FIG. 21 is a functional block diagram showing the configuration of ahead mounted display in a modification.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments of the invention are explained below.

A. First Embodiment A-1. Configuration of a Head-Mounted Display Device

FIG. 1 is an explanatory diagram showing the configuration of theexterior of a head-mounted display in a first embodiment of theinvention. A head-mounted display device HM is a display device mountedon the head and is also called head mounted display (HMD). The headmounted display HM in this embodiment is an optical see-throughhead-mounted display device with which a user can visually recognize avirtual image and at the same time directly visually recognize anexternal scene.

The head mounted display HM includes an image display unit 20 thatcauses the user to visually recognize a virtual image in a state inwhich the image display unit 20 is mounted on the head of the user and acontrol unit (a controller) 10 that controls the image display unit 20.

The image display unit 20 is a mounted member mounted on the head of theuser. In this embodiment, the image display unit 20 has an eyeglassshape. The image display unit 20 includes ear hooking sections 21, aright display driving unit 22, a left display driving unit 24, a rightoptical panel 26, a left optical panel 28, a head-side light emittingunit 61, and a head-side light receiving unit 65. The ear hookingsections 21 are members provided to transverse on the ears of the userfrom the ends of the right display driving unit 22 and the left displaydriving unit 24 and function as temples. The right optical panel 26 andthe left optical panel 28 are arranged to be respectively located beforethe right and left eyes of the user in a state in which the user wearsthe image display unit 20. The right display driving unit 22 is arrangedin a connecting place of the ear hooking section 21 for the right earand the right optical panel 26. The left display driving unit 24 isarranged in a connecting place of the ear hooking section 21 for theleft ear and the left optical panel 28. In the following explanation,the right display driving unit 22 and the left display driving unit 24are collectively referred to simply as “display driving unit”. The rightoptical panel 26 and the left optical panel 28 are collectively referredto simply as “optical panel”.

The head-side light emitting unit 61 and the head-side light receivingunit 65 are arranged to be adjacent to each other on the front surfaceof a housing forming the optical panel (in this embodiment, near anupper part in the center of the right optical panel 26 and the leftoptical panel 28). As the head-side light emitting unit 61 in thisembodiment, an infrared-emitting diode is used. The head-side lightemitting unit 61 emits an infrared ray, which is invisible light. As thehead-side light receiving unit 65 in this embodiment, an infraredphotodiode is used. The head-side light receiving unit 65 receives aninfrared ray.

The display driving unit includes an LCD (Liquid Crystal Display), aprojection optical system, and the like, which are not shown in thefigure. Details are explained later. The optical panel includes a lightguide plate and a light modulating plate, which are not shown in thefigure. The light guide plate is formed of a light transmissive resinmaterial or the like and emits image light, which is captured from thedisplay driving unit, to the eyes of the user. The light modulatingplate is an optical device of a thin plate shape and is arranged tocover the front side (a side opposite to the side of the eyes of theuser) of the light guide plate. The light modulating plate protects thelight guide plate, suppresses, for example, damage and adhesion ofstains to the light guide plate, and adjusts the light transmittance ofthe light modulating plate. Consequently, the light modulating plate canadjust an amount of external light entering the eyes of the user andadjust easiness of visual recognition of a virtual image. The lightmodulating plate can be omitted.

The image display unit 20 further includes a right earphone 32 for theright ear and a left earphone 34 for the left ear. The right earphone 32and the left earphone 34 are respectively worn on the right ear and theleft ear when the user wears the image display unit 20.

The image display unit 20 further includes a connecting unit 40 forconnecting the image display unit 20 to the control unit 10. Theconnecting unit 40 includes a main body cord 48 connected to the controlunit 10, a right cord 42 and a left cord 44, which are two cordsbranching from the main body cord 48, and a coupling member 46 providedat a branch point. The right cord 42 is connected to the right displaydriving unit 22. The left cord 44 is connected to the left displaydriving unit 24. The image display unit 20 and the control unit 10perform transmission of various signals via the connecting unit 40.Connectors (not shown) that fit with each other are respectivelyprovided at an end on the opposite side of the coupling member 46 in themain body cord 48 and the control unit 10. The control unit 10 and theimage display unit 20 are connected and disconnected by fitting theconnector of the main body cord 48 and the connector of the control unit10 and releasing the fitting. As the right cord 42, the left cord 44,and the main body cord 48, for example, a metal cable and an opticalfiber can be adopted.

The control unit 10 is a device for operating the head mounted displayHM. The control unit 10 includes a lighting unit 12, a touch pad 14, across key 16, a power switch 18, a control-unit-side light emitting unit62, and a control-unit-side light receiving unit 66. The lighting unit12 notifies an operation state of the head mounted display HM (e.g., ONor OFF of a power supply) with a light emission state of the lightingunit 12. As the lighting unit 12, for example, an LED (Light EmittingDiode) can be used. The touch pad 14 detects finger operation by theuser on an operation surface (operation panel) of the touch pad 14 andoutputs a signal corresponding to detection content. The cross key 16detects pressing operation of keys corresponding to up, down, left, andright directions and outputs a signal corresponding to detectioncontent. The power switch 18 detects slide operation of the switch toswitch a power-on state of the head mounted display HM. In a housing ofthe control unit 10, a surface on a side on which the main components(i.e., the touch pad 14 and the cross key 16) for the user to performoperation of the control unit 10 are arranged is referred to as“operation surface” as well.

The control-unit-side light emitting unit 62 and the control-unit-sidelight receiving unit 66 are arranged in positions where thecontrol-unit-side light emitting unit 62 and the control-unit-side lightreceiving unit 66 are less easily hidden when the user holds the controlunit 10, for example, near a center line set in the vertical directionon the operation surface of the control unit 10. In this embodiment, thecontrol-unit-side light emitting unit 62 and the control-unit-side lightreceiving unit 66 are arranged on both sides of the lighting unit 12 onthe operation surface. As the control-unit-side light emitting unit 62in this embodiment, an infrared-emitting diode is used. Thecontrol-unit-side light emitting unit 62 emits an infrared ray, which isinvisible light. As the control-unit-side light receiving unit 66 inthis embodiment, an infrared photodiode is used. The control-unit-sidelight receiving unit 66 receives an infrared ray.

FIG. 2 is a functional block diagram showing the configuration of thehead mounted display HM. The control unit 10 includes thecontrol-unit-side light emitting unit 62, a control-unit-side lightreceiving unit 66, an input-information acquiring unit 110, a storingunit 120, a power supply 130, a CPU 140, an interface 180, andtransmitting units (Tx) 51 and 52. These units are connected to oneanother by a not-shown bus.

The input-information acquiring unit 110 has a function of acquiring asignal corresponding to an operation input by the user (e.g., anoperation input to the touch pad 14, the cross key 16, or the powerswitch 18). The storing unit 120 is a storing unit including a ROM, aRAM, a DRAM, and a hard disk, which are not shown in the figure. Thepower supply 130 supplies electric power to the units of the headmounted display HM. As the power supply 130, for example, a secondarybattery can be used.

The CPU 140 executes a computer program installed in advance to therebyprovide a function of an operating system (OS) 150. The CPU 140 expandsfirmware or a computer program stored in the ROM or the hard disk on theRAM and executes the firmware or the computer program to therebyfunction as a direction determining unit 145, an image processing unit160, a sound processing unit 170, and a display control unit 190 aswell. Details of the units are explained later.

The control-unit-side light receiving unit 66 receives an infrared rayemitted from the head-side light emitting unit 61 and outputs an outputsignal corresponding to a signal pattern of the received infrared ray tothe direction determining unit 145. The direction determining unit 145controls driving of the head-side light emitting unit 61, thecontrol-unit-side light emitting unit 62, the head-side light receivingunit 65, and the control-unit-side light receiving unit 66 and executesuser-side see-through processing using output signals from the head-sidelight receiving unit 65 and the control-unit-side light receiving unit66. The user-side see-through processing is processing for erasing avirtual image displayed on the image display unit 20 when it isdetermined that the user faces the operation surface of the control unit10.

The interface 180 is an interface for connecting various externalapparatuses OA (e.g., a personal computer PC, a cellular phone terminal,and a game terminal), which are supply sources of contents, to thecontrol unit 10. As the interface 180, the control unit 10 can include,for example, a USB interface, a micro USB interface, an interface formemory card, or a wireless LAN interface. Contents mean informationcontents including an image (a still image or a moving image) and sound.

The image processing unit 160 generates a clock signal PCLK, a verticalsynchronization signal VSync, a horizontal synchronization signal HSync,and image data Data on the basis of contents input via the interface 180and supplies these signals to the image display unit 20 via theconnecting unit 40. Specifically, the image processing unit 160 acquiresan image signal included in the contents. For example, in the case of amoving image, the acquired image signal is, in general, an analog signalincluding thirty frame images per second. The image processing unit 160separates synchronization signals such as the vertical synchronizationsignal VSync and the horizontal synchronization signal HSync from theacquired image signal. The image processing unit 160 generates the clocksignal PCLK using a not-shown PLL circuit according to periods of thevertical synchronization signal VSync and the horizontal synchronizationsignal HSync separated from the image signal.

The image processing unit 160 converts the analog image signal, fromwhich the synchronization signals are separated, into a digital imagesignal using a not-shown A/D conversion circuit or the like. Thereafter,the image processing unit 160 stores, frame by frame, the digital imagesignal after the conversion in the DRAM in the storing unit 120 as theimage data Data (RGB data) of a target image. The image processing unit160 may execute, on the image data, image processing such as resolutionconversion processing, various kinds of color tone correction processingsuch as adjustment of luminance and chroma, and keystone correctionprocessing according to necessity.

The image processing unit 160 transmits the generated clock signal PCLK,vertical synchronization signal VSync, and horizontal synchronizationsignal HSync and the image data Data stored in the DRAM in the storingunit 120 respectively via the transmitting units 51 and 52. The imagedata Data transmitted via the transmitting unit 51 is referred to as“image data for right eye” as well. The image data Data transmitted viathe transmitting unit 52 is referred to as “image data for left eye” aswell. The transmitting units 51 and 52 function as transceivers forserial transmission between the control unit 10 and the image displayunit 20.

The display control unit 190 generates control signals for controllingthe right display driving unit 22 and the left display driving unit 24.Specifically, the display control unit 190 separately controls,according to the control signals, for example, turn-on and turn-off ofdriving of a right LCD 241 by a right-LCD control unit 211, turn-on andturn-off of driving of a right backlight 221 by a right-backlightcontrol unit 201, turn-on and turn-off of driving of a left LCD 242 by aleft-LCD control unit 212, and turn-on and turn-off of driving of a leftbacklight 222 by the a left-backlight control unit 202 to therebycontrol generation and emission of image light by each of the rightdisplay driving unit 22 and the left display driving unit 24. Forexample, the display control unit 190 causes both the right displaydriving unit 22 and the left display driving unit 24 to generate imagelights, causes only one of the right display driving unit 22 and theleft display driving unit 24 to generate image light, or causes theright display driving unit 22 and the left display driving unit 24 notto generate image light.

The display control unit 190 transmits the control signals for theright-LCD control unit 211 and the left-LCD control unit 212respectively via the transmitting units 51 and 52. The display controlunit 190 transmits the control signals for the right-backlight controlunit 201 and the left-backlight control unit 202.

The sound processing unit 170 acquires a sound signal included in thecontents, amplifies the acquired sound signal, and supplies the soundsignal to the right earphone 32 and the left earphone 34 of the imagedisplay unit 20 via the connecting unit 40.

The image display unit 20 includes the right display driving unit 22,the left display driving unit 24, a right light guide plate 261functioning as the right optical panel 26, a left light guide plate 262functioning as the left optical panel 28, the head-side light emittingunit 61, the head-side light receiving unit 65, the right earphone 32,and the left earphone 34.

The head-side light receiving unit 65 receives an infrared ray emittedfrom the control-unit-side light emitting unit 62 and outputs an outputsignal corresponding to a signal pattern of the received infrared ray tothe direction determining unit 145. The right display driving unit 22includes a receiving unit (Rx) 53, the right-backlight (BL) control unit201 and the right-backlight (BL) 221 functioning as a light source, theright LCD-control unit 211 and the right LCD 241 functioning as adisplay device, and a right projection optical system 251. Theright-backlight control unit 201, the right-LCD control unit 211, theright backlight 221, and the right LCD 241 are collectively referred toas “image-light generating unit” as well.

The receiving unit 53 functions as a transceiver for serial transmissionbetween the control unit 10 and the image display unit 20. Theright-backlight control unit 201 has a function of driving the rightbacklight 221 on the basis of an input control signal. The rightbacklight 221 is a light emitting member such as an LED. The rightLCD-control unit 211 has a function of driving the right LCD 241 on thebasis of the clock signal PCLK, the vertical synchronization signalVSync, the horizontal synchronization signal HSync, and the image datafor right eye input via the receiving unit 53. The right LCD 241 is atransmissive liquid crystal panel in which plural pixels are arranged ina matrix shape.

FIG. 3 is an explanatory diagram showing a state in which image light isemitted by the image-light generating unit. The right LCD 241 has afunction of driving liquid crystals corresponding to the positions ofthe pixels arranged in the matrix shape to thereby change thetransmittance of light transmitted through the right LCD 241 to modulateillumination light IL irradiated from the right backlight 221 intoeffective image light PL representing an image. As shown in FIG. 3, inthis embodiment, the backlight system is adopted. However, image lightmay be emitted using a front light system or a reflection system.

The right projection optical system 251 shown in FIG. 2 includes acollimate lens that converts image light emitted from the right LCD 241into light beams in a parallel state. The right light guide plate 261functioning as the right optical panel 26 guides the image light outputfrom the right projection optical system 251 to the right eye of theuser while reflecting the image light along a predetermined opticalpath. The right projection optical system 251 and the right light guideplate 261 are collectively referred to as “light guide unit” as well.

The left display driving unit 24 includes a receiving unit (Rx) 54, theleft-backlight (BL) control unit 202 and the left-backlight (BL) 222functioning as a light source, the left-LCD control unit 212 and theleft LCD 242 functioning as a display device, and a left projectionoptical system 252. The left-backlight control unit 202, the left-LCDcontrol unit 212, the left backlight 222, and the left LCD 242 arecollectively referred to as “image light generating unit” as well. Theleft projection optical system 252 and the left light guide plate 262are collectively referred to as “light guide unit” as well. The rightdisplay driving unit 22 and the left display driving unit 24 form apair. The units of the left display driving unit 24 have configurationsand perform operations same as those of the units of the right displaydriving unit 22 explained below. Therefore, detailed explanation of theunits of the left display driving unit 24 is omitted.

FIG. 4 is an explanatory diagram showing an example of a virtual imagerecognized by the user. The image lights guided to the eyes of the userof the head mounted display HM are focused on the retinas of the user asexplained above, whereby the user can visually recognize a virtualimage. As shown in FIG. 4, a virtual image VI is displayed in a visualfield VR of the user of the head mounted display HM. In the visual fieldVR of the user except a portion where the virtual image VI is displayedin the visual field VR, the user can see an external scene SC throughthe right optical panel 26 and the left optical panel 28. In the headmounted display HM in this embodiment, in the portion where the virtualimage VI is displayed in the visual field VR of the user, the user canalso see the external scene SC in the background of the virtual imageVI.

A-2. User-Side See-Through Processing

FIG. 5 is a flowchart for explaining a procedure of the user-sidesee-through processing for the head mounted display HM. The user-sidesee-through processing is processing for erasing a virtual imagedisplayed on the image display unit 20 when it is detected that the userfaces the operation surface of the control unit 10, i.e., it is detectedthat the operation surface of the control unit 10 and the image displayunit 20 are opposed to each other because the user attempts to operatethe control unit 10. The user-side see-through processing is executed atany time after the head mounted display HM is started.

The direction determining unit 145 of the control unit 10 causes thehead-side light emitting unit 61 and the control-unit-side lightemitting unit 62 to emit lights (step S102). Specifically, the directiondetermining unit 145 of the control unit 10 alternately drives thehead-side light emitting unit 61 and the control-unit-side lightemitting unit 62 to thereby cause the head-side light emitting unit 61and the control-unit-side light emitting unit 62 to alternately emitinfrared rays. By causing the head-side light emitting unit 61 and thecontrol-unit-side light emitting unit 62 to alternately emit infraredrays in this way, it is possible to suppress interference of theinfrared ray from the head-side light emitting unit 61 and the infraredray from the control-unit-side light emitting unit 62. A unit time intime division can be arbitrarily set.

The direction determining unit 145 determines whether the head-sidelight receiving unit 65 and the control-unit-side light receiving unit66 receive the infrared rays (step S104). Specifically, the directiondetermining unit 145 determines whether the direction determining unit145 receives both an output signal from the head-side light receivingunit 65 and an output signal from the control-unit-side light receivingunit 66 within a predetermined time. When the direction determining unit145 receives both the output signals, the direction determining unit 145determines that the head-side light receiving unit 65 and thecontrol-unit-side light receiving unit 66 receive the infrared rays (YESin step S104). When the direction determining unit 145 does not receiveat least one of the output signals, the direction determining unit 145determines that the head-side light receiving unit 65 and thecontrol-unit-side light receiving unit 66 do not receive the infraredrays (No in step S104). The predetermined time can be arbitrarily set.However, it is desirable to set the predetermined time twice or more aslong as the unit time in step S102.

When the head-side light receiving unit 65 and the control-unit-sidelight receiving unit 66 do not receive the infrared rays (No in stepS104), the direction determining unit 145 shifts the processing to stepS102 and causes the head-side light emitting unit 61 and thecontrol-unit-side light emitting unit 62 to emit lights. On the otherhand, when the head-side light receiving unit 65 and thecontrol-unit-side light receiving unit 66 receive the infrared rays (YESin step S104), the direction determining unit 145 turns off thebacklights (step S106). Specifically, the direction determining unit 145requests the display control unit 190 of the control unit 10 to turn offthe backlights.

FIG. 6 is an explanatory diagram showing a state of the image-lightgenerating unit in step S106 in FIG. 5. In step S106 in FIG. 5, thedisplay control unit 190 that receives the request from the directiondetermining unit 145 transmits a control signal indicating turn-off ofthe driving of the right backlight 221 by the right-backlight controlunit 201 and a control signal indicating turn-off of the driving of theleft backlight 222 by the left-backlight control unit 202 to the imagedisplay unit 20. The right-backlight control unit 201 that receives thesignal turns off the right backlight 221. Similarly, the left-backlightcontrol unit 202 that receives the signal turns off the left backlight222. As a result, as shown in FIG. 6, images rendered on the right LCD241 and the left LCD 242 are not emitted as image lights. Therefore, thedisplay of the virtual image VI disappears from the visual field VR ofthe user.

After turning off the backlights, the direction determining unit 145determines whether the head-side light receiving unit 65 and thecontrol-unit-side light receiving unit 66 continue the reception of theinfrared rays (step S108). The determination in step S108 is performedby the same method as step S104.

When the head-side light receiving unit 65 and the control-unit-sidelight receiving unit 66 continue the reception of the infrared rays (YESin step S108), the direction determining unit 145 shifts the processingto step S106 and continues the turned-off state of the backlights. Onthe other hand, when at least one of the head-side light receiving unit65 and the control-unit-side light receiving unit 66 do not receive theinfrared ray (YES in step S104), the direction determining unit 145turns on the backlights (step S110). Specifically, the directiondetermining unit 145 requests the display control unit 190 of thecontrol unit 10 to turn on the backlights and ends the processing.

In step S110 in FIG. 5, the display control unit 190 that receives therequest from the direction determining unit 145 transmits a controlsignal indicating turn-on of the driving of the right backlight 221 bythe right-backlight control unit 201 and a control signal indicatingturn-on of the driving of the left backlight 222 by the left-backlightcontrol unit 202 to the image display unit 20. The right-backlightcontrol unit 201 that receives the signal turns on the right backlight221. Similarly, the left-backlight control unit 202 that receives thesignal turns on the left backlight 222. As a result, images rendered onthe right LCD 241 and the left LCD 242 are emitted as image lights. Thevirtual image VI is displayed in the visual field VR of the user again.

FIGS. 7A and 7B are explanatory diagrams showing states in which theuser-side see-through processing (FIG. 5) is executed. FIG. 7A shows astate in which the user wearing the head mounted display HM faces aplace other than the operation surface of the control unit 10. When theuser wearing the head mounted display HM faces a place other than theoperation surface of the control unit 10, both an infrared ray CAemitted from the head-side light emitting unit 61 of the image displayunit 20 and an infrared ray MA emitted from the control-unit-side lightemitting unit 62 of the control unit 10 are not received (No in stepS104 in FIG. 5). Therefore, since the turn-off of the backlights in theuser-side see-through processing (step S106) is not performed, thevirtual image VI is displayed in the visual field VR of the user.

FIG. 7B shows a state in which the user wearing the head mounted displayHM faces the operation surface of the control unit 10. When the userwearing the head mounted display HM faces the operation surface of thecontrol unit 10, the infrared ray CA emitted from the head-side lightemitting unit of the image display unit 20 is received by thecontrol-unit-side light receiving unit 66 of the control unit 10.Similarly, the infrared ray MA emitted from the control-unit-side lightemitting unit 62 of the control unit 10 is received by the head-sidelight receiving unit 65 of the image display unit 20 (YES in step S104in FIG. 5). Therefore, the turn-off of the backlights in the user-sidesee-through processing (step S106) is performed and the display of thevirtual image VI disappears from the visual field VR of the user. As aresult of the disappearance of the virtual image VI that blocks thevisual field VR, the user can clearly see an external scene, i.e., theoperation surface of the control unit 10.

The state in which the user wearing the head mounted display HM facesthe operation surface of the control unit 10 as shown in FIG. 7B, i.e.,a state in which an optical panel surface of the image display unit 20and the operation surface of the control unit 10 are opposed to eachother is expressed as “the operation surface and the image display unit20 are opposed to each other” as well.

When the right backlight 221 and the left backlight 222 include plurallight emitting members (LEDs, etc.), for example, processing explainedbelow may be performed in step S106 of the user-side see-throughprocessing (FIG. 5).

FIG. 8 is an explanatory diagram showing another processing form in stepS106 of the user-side see-through processing (FIG. 5). The rightbacklight 221 and the left backlight 222 shown in FIG. 8 are lightsources, each including twelve LEDs. In step S106 in FIG. 5, thedirection determining unit 145 requests the display control unit 190 ofthe control unit 10 to turn off a part of the LEDs. The display controlunit 190 that receives the request transmits identifiers for identifyingthe LEDs and control signals for designating turn-on and turn-off ofdriving of the LEDs to the right-backlight control unit 201 and theleft-backlight control unit 202. The right-backlight control unit 201that receives the signals turns off a designated part of the LEDs of theright backlight 221. Similarly, the left-backlight control unit 202turns off a designated part of the LEDs of the left backlight 222.

As a result, as shown in FIG. 8, in images rendered on the right LCD 241and the left LCD 242, images corresponding to a portion of a region PTin which the turned-off LEDs are arranged are not emitted as imagelights. Therefore, the virtual image VI in another portion excluding theregion PT is displayed in the visual field VR of the user. Consequently,the virtual image VI in a portion corresponding to the region PT inwhich the turned-off LEDs are arranged is not displayed. Therefore, itis possible to improve the visibility of an external scene in theportion of the region PT.

In FIG. 8, an operation explanation and a function explanation for thecontrol unit 10 may be displayed in the other portion excluding theregion PT. Then, the user can check the operation surface on the controlunit 10 side referring to the operation explanation and the functionexplanation while keeping wearing the head mounted display HM.Therefore, it is possible to improve convenience.

Further, in step S106 of the user-side see-through processing (FIG. 5),the direction determining unit 145 may perform, for example, processingexplained below instead of turning off the backlights.

In step S106 in FIG. 5, the direction determining unit 145 requests thedisplay control unit 190 of the control unit 10 to reduce the luminanceof the backlights. The display control unit 190 that receives therequest transmits, together with a control signal for designatingturn-on and turn-off of driving of the backlights, a control signal fordesignating the luminance of the backlights to the right-backlightcontrol unit 201 and the left-backlight control unit 202. As the controlsignal for designating the luminance of the backlights, for example, aPWM (Pulse Width Modulation) signal can be used. Consequently, in stepS106, a reduction of the illumination light by the backlights (the rightbacklight 221 and the left backlight 222) is performed instead of theturn-off of the backlights. If the illumination light is reduced, sincethe image light emitted by the image-light generating unit becomesfeeble (the luminance of the image-light generating unit decreases), thevirtual image VI displayed in the visual field VR of the user isdisplayed pale and blurred. Therefore, the user can easily visuallyrecognize an external scene, i.e., the operation surface of the controlunit 10.

In step S106 in FIG. 5, the direction determining unit 145 requests thedisplay control unit 190 of the control unit 10 to reduce an apertureratio of the LCDs (liquid crystals). The display control unit 190 thatreceives the request transmits a control signal for designating anaperture ratio of the LCDs to the right-LCD control unit 211 and theleft-LCD control unit 212. Consequently, in step S106, a reduction ofthe aperture ratio of the LCDs (the right LCD 241 and the left LCD 242)is performed instead of the turn-off of the backlights. If the liquidcrystal aperture ratio is reduced, since the image light emitted by theimage-light generating unit becomes feeble, the virtual image VIdisplayed in the visual field VR of the user is displayed pale andblurred. Therefore, the user can easily visually recognize an externalscene, i.e., the operation surface of the control unit 10.

In step S106 in FIG. 5, the direction determining unit 145 requests theimage processing unit 160 of the control unit 10 to change the imagedata Data to dummy data of a single color black. The image processingunit 160 that receives the request changes the image data Data to betransmitted to the image display unit 20 to the dummy data of the singlecolor black. Consequently, in step S106, images rendered on the LCDs(the right LCD 241 and the left LCD 242) are adjusted to dummy images ofthe single color black. The image-light generating unit emits imagelight corresponding to the dummy images. Therefore, the virtual image VIdisplayed in the visual field VR of the user is displayed as if thevirtual image VI disappears. Therefore, the user can easily visuallyrecognize an external scene, i.e., the operation surface of the controlunit 10. The image processing unit 160 may replace at least a part ofthe image data Data with the dummy data of the single color black ratherthan the entire image data Data.

As explained above, according to the first embodiment, the directiondetermining unit 145 assumes that the user shifts the user's attentionfrom the virtual image. Specifically, the direction determining unit 145determines whether the user faces the operation surface of the controlunit 10. When the direction determining unit 145 determines that theuser faces the operation surface, the control unit 10 adjusts theluminance of the image-light generating unit or adjusts the image lightPL generated by the image-light generating unit to reduce the visibilityof the virtual image VI. Therefore, in the head mounted display HM, itis possible to detect, with a configuration different from that in thepast, that the user attempts to operate the control unit 10 of the headmounted display HM. It is possible to improve the visibility of theexternal scene SC in the image display unit 20.

Specifically, the head-side light emitting unit 61 that emits invisiblelight (an infrared ray) is arranged in the image display unit 20. Thecontrol-unit-side light receiving unit 66 that receives the emittedinvisible light is arranged on the operation surface of the control unit10. The control-unit-side light emitting unit 62 that emits invisiblelight (an infrared ray) is further arranged on the operation surface ofthe control unit 10. The head-side light receiving unit 65 that receivesthe emitted invisible light is further arranged in the image displayunit 20. The control unit 10 causes the head-side light emitting unit 61and the control-unit-side light emitting unit 62 to alternately emit theinvisible light. The direction determining unit 145 determines, using anoutput signal of the control-unit-side light receiving unit 66 and anoutput signal of the head-side light receiving unit 65, whether theoperation surface of the control unit 10 and the image display unit 20are opposed to each other. When the user attempts to operate the controlunit 10 of the head mounted display HM, the user looks at the operationsurface of the control unit 10, i.e., the user faces the operationsurface (i.e., the image display unit 20 mounted on the head of the userand the operation surface are opposed to each other). Therefore, thedirection determining unit 145 can determine whether the user faces theoperation surface by determining whether the operation surface of thecontrol unit 10 and the image display unit 20 are opposed to each otheras explained above.

When the direction determining unit 145 determines that the user facesthe operation surface (i.e., the operation surface of the control unit10 and the image display unit 20 are opposed to each other), by turningoff or reducing the illumination light IL of the light sources (theright backlight 221 and the left backlight 222), the control unit 10 canadjust the luminance of the image-light generating unit to reduce thevisibility of the virtual image VI. When the direction determining unit145 determines that the user faces the operation surface (i.e., theoperation surface of the control unit 10 and the image display unit 20are opposed to each other), by replacing at least a part of the imagedata Data to be transmitted to the image-light generating unit withdummy data indicating black, the control unit 10 can adjust the imagelight PL generated by the image-light generating unit to reduce thevisibility of the virtual image VI. When the direction determining unit145 determines that the user faces the operation screen (i.e., theoperation surface of the control unit 10 and the image display unit 20are opposed to each other), by reducing a liquid crystal aperture ratioof at least a part of the display devices (the right LCD 241 and theleft LCD 242), the control unit 10 can adjust the image light PLgenerated by the image-light generating unit to reduce the visibility ofthe virtual image VI. In this way, the visibility of the virtual imageVI that blocks the visual field VR of the user is reduced. As a result,the visibility of an external scene in the image display unit 20 can beimproved. The user can clearly see the external scene, i.e., theoperation surface of the control unit 10.

Further, according to this embodiment, infrared ray emitting units thatemit infrared rays are used as the head-side light emitting unit 61 andthe control-unit-side light emitting unit 62 and infrared ray receivingunits that receive the emitted infrared rays are used as the head-sidelight receiving unit 65 and the control-unit-side light receiving unit66. Therefore, it is possible to realize, at low cost, the head mounteddisplay HM in which the visibility of an external scene in the imagedisplay unit 20 is improved with a configuration different from that inthe past.

Further, according to this embodiment, sets of light emitting units andlight receiving units are respectively arranged in the control unit 10and the image display unit 20. The control unit 10 causes two lightemitting units (the head-side light emitting unit 61 and thecontrol-unit-side light emitting unit 62) to alternately emit invisiblelight (infrared rays). The direction determining unit 145 determines,using output signals from the two light receiving units (thecontrol-unit-side light receiving unit 66 and the head-side lightreceiving unit 65), whether the operation surface of the control unit 10and the image display unit 20 are opposed to each other. Therefore, itis possible to suppress wrong determination and improve accuracy of thedetermination by the direction determining unit 145.

B. Second Embodiment

In a second embodiment of the invention, a configuration that canimprove reliability and power saving properties in user-side see-throughprocessing is explained. In the following explanation, only componentshaving configurations and operations different from those in the firstembodiment are explained. In the figures, components same as those inthe first embodiment are denoted by reference numerals and signs same asthose in the first embodiment explained above and detailed explanationof the components is omitted.

B-1. Configuration of a Head-Mounted Display Device

FIG. 9 is an explanatory diagram showing the configuration of theexterior of a head mounted display HMa in the second embodiment. FIG. 10is a functional block diagram showing the configuration of the headmounted display HMa in the second embodiment. The head mounted displayHMa is different from the head mounted display HM in the firstembodiment shown in FIGS. 1 and 2 in that the head mounted display HMaincludes a control unit 10 a instead of the control unit 10 and includesan image display unit 20 a instead of the image display unit 20.

The image display unit 20 a includes a head-side light emitting unit 61a instead of the head-side light emitting unit 61. The head-side lightemitting unit 61 a has a function of emitting an infrared ray, which isinvisible light. The head-side light emitting unit 61 a emits aninfrared ray having a signal pattern including identificationinformation of the image display unit 20 a by pulse-modulating aninfrared ray to be emitted. The identification information isinformation including an arbitrary character string for identifying theimage display unit 20 a.

The control unit 10 a includes a direction determining unit 145 ainstead of the direction determining unit 145 and includes a contactdetecting unit 71. Authentication information CI is stored in thestoring unit 120 of the control unit 10 a. The contact detecting unit 71is arranged on a surface on the opposite side of an operation surface ofthe control unit 10 a and in a portion (e.g., the center or the left andright ends) with which the hand of the user is assumed to come intocontact when the user performs operation holding the control unit 10 a.The contact detecting unit 71 in this embodiment includes a touch sensorand detects contact by the user. As the touch sensor, touch sensors ofvarious systems can be adopted. The direction determining unit 145 aincludes an authenticating unit 146. Details of the authenticating unit146 are explained later. In the authentication information CI stored inthe storing unit 120, identification information of the image displayunit 20 a connected to the control unit 10 a is stored in advance.

B-2. User-Side See-Through Processing

FIG. 11 is a flowchart for explaining a procedure of the user-sidesee-through processing for the head mounted display HMa in the secondembodiment. The flowchart is different from the flowchart in the firstembodiment shown in FIG. 5 in that the flowchart further includes stepsS202 and S204. The other actions are the same as those in the firstembodiment. First, the direction determining unit 145 a determines,according to an output signal from the contact detecting unit 71,whether contact with the control unit 10 a is detected (step S202). Whenthe contact is not detected, the direction determining unit 145 a shiftsthe processing to step S202. On the other hand, when the contact isdetected the direction determining unit 145 a shifts the processing tostep S204 and drives the head-side light emitting unit 61 a and thecontrol-unit-side light emitting unit 62.

When the direction determining unit 145 a determines in step S104 thatthe direction determining unit 145 a receives both an output signal fromthe head-side light receiving unit 65 and an output signal from thecontrol-unit-side light receiving unit 66, the authenticating unit 146of the direction determining unit 145 a performs authentication of theimage display unit 20 a (step S204). Specifically, the authenticatingunit 146 performs the authentication of the image display unit 20 a byperforming a search as to whether identification information of theimage display unit 20 a acquired from the output signal from thecontrol-unit-side light receiving unit 66 coincides with theidentification information in the authentication information CI storedin the storing unit 120. When the identification information included inthe output signal coincides with the identification information in theauthentication information CI, the authenticating unit 146 determinesthat the authenticating unit 146 succeeds in the authentication of theimage display unit 20 a (YES in step S204). On the other hand, when theidentification information included in the output signal does notcoincide with the identification information in the authenticationinformation CI, the authenticating unit 146 determines that theauthenticating unit 146 fails in the authentication of the image displayunit 20 a (NO in step S204). When the authenticating unit 146 fails inthe authentication, the direction determining unit 145 a shifts theprocessing to step S102 and causes the head-side light emitting unit 61a and the control-unit-side light emitting unit 62 to emit lights. Onthe other hand, when the authenticating unit 146 succeeds in theauthentication, the direction determining unit 145 a shifts theprocessing to step S106 and turns off the backlights.

As explained above, according to the second embodiment, the contactdetecting unit 71 for detecting contact with the control unit 10 a isarranged in the control unit 10 a. When the contact with the controlunit 10 a is detected, the direction determining unit 145 a of thecontrol unit 10 a causes the head-side light emitting unit 61 a to emitan infrared ray and causes the control-unit-side light emitting unit 62to emit an infrared ray. When the user attempts to operate the headmounted display HMa, in general, the user holds the control unit 10 a.Therefore, according to the second embodiment, it is possible to reducepower consumption of the head mounted display HMa compared with theconfiguration in which infrared rays are always emitted by the lightemitting units (the head-side light emitting unit 61 and thecontrol-unit-side light emitting unit 62).

Further, according to the second embodiment, the identificationinformation for identifying the image display unit 20 a is included inthe infrared ray emitted by the head-side light emitting unit 61 a. Theauthenticating unit 146 can authenticate the image display unit 20 a byacquiring the identification information for the image display unit 20 afrom the output signal of the control-side light receiving unit andperforming a search as to whether the acquired identificationinformation is included in the authentication information CI stored inadvance. Therefore, it is possible to suppress an infrared ray emittedfrom another apparatus, which can emit an infrared ray, from beingdetected by mistake (misrecognition) in a public space or the like andimprove reliability in the user-side see-through processing. Asexplained above, according to the second embodiment, in the head mounteddisplay HMa, it is possible to further improve reliability and powersaving properties of the user-side see-through processing.

C. Third Embodiment

In a third embodiment of the invention, a configuration for performing,instead of the user-side see-through processing, movement detectionprocessing for detecting the movement of the head exceeding a fixedamount of a user wearing an image display unit is explained. In thefollowing explanation, only components having configurations andoperations different from those in the first embodiment are explained.In the figures, components same as those in the first embodiment aredenoted by reference numerals and signs same as those in the firstembodiment explained above and detailed explanation of the components isomitted.

C-1. Configuration of a Head-Mounted Display Device

FIG. 12 is an explanatory diagram showing the configuration of theexterior of a head-mounted display device according to the thirdembodiment. FIG. 13 is a functional block diagram of the configurationof a head mounted display HMb in the third embodiment. The head mounteddisplay HMb is different from the head mounted display HM in the firstembodiment shown in FIGS. 1 and 2 in that the head mounted display HMbincludes a control unit 10 b instead of the control unit 10 and includesan image display unit 20 b instead of the image display unit 20.

In the control unit 10 b, threshold information CII is stored in thestoring unit 120 instead of the authentication information CI. Thethreshold information CII is a threshold used in movement detectionprocessing (explained later). In this embodiment, two thresholds (afirst threshold and a second threshold) are stored in advance. Thecontrol unit 10 b includes, in the CPU 140, a detecting unit 147 insteadof the direction determining unit 145. The control unit 10 b does notinclude the control-unit-side light emitting unit 62 and thecontrol-unit-side light receiving unit 66.

The detecting unit 147 acquires change information (in this embodiment,an angular velocity, which is a detection value of a gyro sensor 74),indicating a change in the direction of the image display unit 20 b, andexecutes the movement detection processing using the change information.The movement detection processing is processing for detecting themovement of the head exceeding a fixed amount of a user wearing theimage display unit 20 b of the head mounted display HMb and erasing avirtual image displayed on the image display unit 20 b. Details of themovement detection processing are explained later. The detecting unit147 corresponds to “detecting unit” in the appended claims.

The image display unit 20 a includes a gyro sensor 74 and a settingbutton 72 instead of the head-side light emitting unit 61 and thehead-side light receiving unit 65.

The gyro sensor 74 functioning as an angular-velocity detecting unit isarranged on the inside of a housing of the right display unit 22. Thegyro sensor 74 in this embodiment is a two-axis angular velocity sensorof a piezoelectric vibration type. The gyro sensor 74 detects angularvelocities on two axes (an x axis and a y axis) of the image displayunit 20 b. The setting button 72 is arranged on the surface of a housingof the left display unit 24, which is the surface on the outer side ofthe image display unit 20 b (i.e., a surface on the opposite side of amounting side of the image display unit 20 b). The setting button 72 isused for setting an initial position that should be a reference indetecting the direction of the image display unit 20 b in the movementdetection processing explained later. Details of the setting button 72are explained later.

C-2. Movement Detection Processing

FIGS. 14A and 14B are explanatory diagrams for explaining the movementdetection processing. The movement detection processing for the headmounted display HMb is processing for detecting the movement of the headexceeding a fixed amount of a user wearing the image display unit 20 band erasing a virtual image displayed on the image display unit 20 b.The “movement of the head exceeding a fixed amount” means that the headof the user wearing the image display unit 20 b (i.e., the image displayunit 20 b) moves exceeding a fixed amount with respect to the initialposition. In this embodiment, the initial position and a movement amountof the movement of the head are specified by a combination of the angleof the head and the direction of the face of the user wearing the imagedisplay unit 20 b at the time when the setting button 72 is pressed.

As shown in FIG. 14A, the angle of the head corresponds to the movementof the head in a perpendicular direction (a vertical direction) PL ofthe user. For example, when the setting button 72 is pressed when thehead of the user is in a perpendicular state (0 degree), 0 degree is setas the initial position. In this state, when the user turns the head toan MV direction, a movement amount of the movement of the head (an angleθ1 of the head) is 45 degrees, which is a difference between the initialposition (0 degree) and a position (45 degrees) after the movement. Themovement amount (the angle θ1 of the head) of the movement of the headcorresponds to the angular velocity of the y axis obtained from anoutput value of the gyro sensor 74.

As shown in FIG. 14B, the direction of the face corresponds to themovement of the face in the horizontal direction HL of the user. Forexample, when the setting button 72 is pressed when the face of the userfaces the front (0 degree), 0 degree is set as the initial position. Inthis state, when the user turns the face to the MV direction, a movementamount of the movement of the head (a direction θ2 of the face) is 70degrees, which is a difference between the initial position (0 degree)and a position (70 degrees) after the movement. The movement amount ofthe movement of the head (the direction θ2 of the face) corresponds tothe angular velocity of the x axis obtained from an output value of thegyro sensor 74.

FIG. 15 is a flowchart for explaining a procedure of the movementdetection processing. The detecting unit 147 determines whether thesetting button 72 provided in the image display unit 20 b is pressed(step S402). When the setting button 72 is not pressed (No in stepS402), the detecting unit 147 transitions to step S402 and continues tomonitor whether the setting button 72 is pressed. When the settingbutton 72 is pressed (YES in step S402), the detecting unit 147 readsthe first threshold and the second threshold from the thresholdinformation CII stored in the storing unit 120 (step S404).

After acquiring the thresholds, the detecting unit 147 acquires anoutput value of the gyro sensor 74 (step S406). The detecting unit 147determines whether at least one of the angle of the x axis and the angleof the y axis obtained from the acquired output value exceeds a firstthreshold Th1 (step S410). The detecting unit 147 integrates the angularvelocity of the x axis and the angular velocity of the y axis tocalculate the angle of the x axis and the angle of the y axis. When boththe angles of the x axis and the y axis are equal to or smaller than thefirst threshold Th1 (NO in step S410), the detecting unit 147 shifts theprocessing to step S406 and acquires an output value of the gyro sensor74 again.

On the other hand, when at least one of the angles of the x axis and they axis exceeds the first threshold Th1 (YES in step S410), the detectingunit 147 determines whether at least one of the angular velocity of thex axis and the angular velocity of the y axis exceeds a second thresholdTh2 (step S412). When both the angular velocities of the x axis and they axis are equal to or smaller than the second threshold Th2 (NO in stepS412), the detecting unit 147 shifts the processing to step S406 andacquires an output value of the gyro sensor 74 again.

On the other hand, when at least one of the angular velocities of the xaxis and the y axis exceeds the second threshold Th2 (YES in step S412),the detecting unit 147 turns off the backlights (step S414).Specifically, the detecting unit 147 requests the display control unit190 of the control unit 10 b to turn off the backlights.

FIG. 16 is an explanatory diagram showing a state of the image-lightgenerating unit in step S414 in FIG. 15. In step S414 in FIG. 15, thedisplay control unit 190 that receives the request from the detectingunit 147 transmits a control signal indicating turn-off of the drivingof the right backlight 221 by the right-backlight control unit 201 and acontrol signal indicating turn-off of the driving of the left backlight222 by the left-backlight control unit 202 to the image display unit 20b. The right-backlight control unit 201 that receives the signal turnsoff the right backlight 221. Similarly, the left-backlight control unit202 that receives the signal turns off the left backlight 222. As aresult, as shown in FIG. 16, since images rendered on the right LCD 241and the left LCD 242 are not emitted as image lights, the display of thevirtual image VI disappears from the visual field VR of the user.

After turning off the backlights, the detecting unit 147 acquires anoutput value of the gyro sensor 74 (step S416). The detecting unit 147determines whether at least one of the angle of the x axis and the angleof the y axis obtained from the acquired output value is equal to orlarger than the first threshold Th1 (step S418). When both the angles ofthe x axis and the y axis are smaller than the first threshold Th1 (NOin step S418), the detecting unit 147 shifts the processing to step S416and acquires an output value of the gyro sensor 74 again.

On the other hand, when at least one of the angles of the x axis and they axis is equal to or larger than the first threshold Th1 (YES in stepS418), the detecting unit 147 turns on the backlights (step S420).Specifically, the detecting unit 147 requests the display control unit190 of the control unit 10 b to turn on the backlights and ends theprocessing.

In step S420 in FIG. 15, the display control unit 190 that receives therequest from the detecting unit 147 transmits a control signalindicating turn-on of the driving of the right backlight 221 by theright-backlight control unit 201 and a control signal indicating turn-onof the driving of the left backlight 222 by the left-backlight controlunit 202 to the image display unit 20 b. The right-backlight controlunit 201 that receives the signal turns on the right backlight 221.Similarly, the left-backlight control unit 202 that receives the signalturns on the left backlight 222. As a result, images rendered on theright LCD 241 and the left LCD 242 are emitted as image lights. Thevirtual image VI is displayed in the visual field VR of the user again.

In this way, according to the pressing of the setting button 72, stepS404 and subsequent steps of the movement detection processing (FIG. 15)are started and the monitoring of an output value of the gyro sensor 74is started. The gyro sensor 74 is a sensor that detects angularvelocity, i.e., a change amount of an angle (i.e., a change in adirection) per a unit time. Therefore, it can be said that the detectingunit 147 sets, according to the pressing of the setting button 72, theangle of the head and the direction of the face of the user at thatpoint in the initial position of the image display unit 20 b (a positionserving as a reference in detecting the movement of the image displayunit 20 b) and the gyro sensor 74 detects angular velocity indicating achange in the direction of the image display unit 20 b with respect tothe initial position. The angular velocity detected by the gyro sensor74 corresponds to “change information” in the appended claims.

The first and second thresholds stored in the threshold information CIIin advance can be arbitrarily set. The first threshold serving as athreshold for an angle can be set to, for example, 45 degrees. Bylimiting a change in an angle with the first threshold, in the movementdetection processing, it is possible to detect “the movement of the headexceeding a fixed amount” of the user wearing the image display unit 20b. In other words, when a change in the angle of the image display unit20 b occurs, it is possible to determine whether the user turns theuser's face away from the initial position and the movement of the headexceeding the fixed amount occurs or the head of the user only slightlymoves.

The second threshold serving as a threshold for angular velocity (achange amount of an angle per a unit time) is desirably set to a smallvalue. By limiting a change in angular velocity with the secondthreshold, in the movement detection processing, it is possible toneglect (allow) slow movement of the user wearing the image display unit20 b and continue the display of the virtual image VI. The user may beallowed to arbitrary change the threshold information CII.

FIGS. 17A and 17B are explanatory diagrams showing states in which themovement detection processing is executed. FIG. 17A shows a state inwhich the user wearing the image display unit 20 b of the head mounteddisplay HMb presses the setting button 72. After the user presses thesetting button 72, when angles and angular velocities obtained from anobtained output value of the gyro sensor 74 do not exceed the first andsecond thresholds, in other words, when the conditions in steps S410 toS412 are not satisfied, the turn-off of the backlights (step S414) inthe movement detection processing is not performed. Therefore, as shownin FIG. 17A, the virtual image VI is displayed in the visual field VR ofthe user.

FIG. 17B shows a state in which the user wearing the image display unit20 b of the head mounted display HMb turns the user's face away from aninitial position. When the user looks, for example, sideways from theinitial position shown in FIG. 17A, changes occur in the angle and theangular velocity in the x axis direction calculated from an output valueof the gyro sensor 74. When the angle and the angular velocity exceedthe first and second thresholds and the conditions in steps S410 to S412are satisfied, the turn-off of the backlights (step S414) in themovement detection processing is performed and the display of thevirtual image VI disappears from the visual field VR of the user. As aresult of the disappearance of the display of the virtual image VI thatblocks the visual field VR, the user can clearly see an external scene.The same holds true when the user looks, for example, upward ordownward, whereby changes occur in the angle and the angular velocity inthe y axis direction calculated from an output value of the gyro sensor74.

Further, in step S414 of the movement detection processing (FIG. 15),the detecting unit 147 may perform, for example, processing explainedbelow instead of turning-off the backlights.

In step S414 in FIG. 15, the detecting unit 147 requests the displaycontrol unit 190 of the control unit 10 b to reduce the luminance of thebacklights. The display control unit 190 that receives the requesttransmits a control signal for designating the luminance of thebacklights to the right-backlight control unit 201 and theleft-backlight control unit 202 together with a control signal fordesignating turn-on and turn-off of the driving of the backlights. Asthe control signal for designating the luminance of the backlights, forexample, a PWM (Pulse Width Modulation) signal can be used.Consequently, in step S406, a reduction of the illumination light by thebacklights (the right backlight 221 and the left backlight 222) isperformed instead of the turn-off of the backlights. If the illuminationlight is reduced, the image light emitted by the image-light generatingunit becomes feeble (the luminance of the image-light generating unitdecreases). Therefore, the virtual image VI displayed in the visualfield VR of the user is displayed pale and blurred. As a result, theuser can easily visually recognize an external scene.

In step S414 in FIG. 15, the detecting unit 147 requests the displaycontrol unit 190 of the control unit 10 b to temporarily stop thedriving of the LCDs (liquid crystals). The display control unit 190 thatreceives the request transmits a control signal for designating turn-offof the driving of the LCDs to the right-LCD control unit 211 and theleft-LCD control unit 212. Consequently, the rendering of images by theLCDs (the right LCD 241 and the left LCD 242) is stopped and thegeneration and the emission of image light in the image-light generatingunit are stopped. Therefore, the display of the virtual image VIdisappears from the visual field VR of the user. As a result, the usercan easily visually recognize an external scene.

The detecting unit 147 may request the display control unit 190 of thecontrol unit 10 b to reduce an aperture ratio of the LCDs (liquidcrystals). If the liquid crystal aperture ratio is reduced, since theimage light emitted by the image-light generating unit becomes feeble,the virtual image VI displayed in the visual field VR of the user isdisplayed pale and blurred. Therefore, the user can easily visuallyrecognize an external scene.

In step S414 in FIG. 15, the detecting unit 147 requests the imageprocessing unit 160 of the control unit 10 b to temporarily stop thetransmission of the image data Data. The image processing unit 160 thatreceives the request stops the image data Data to be transmitted to theimage display unit 20 b. Consequently, the rendering of images by theLCDs (the right LCD 241 and the left LCD 242) is stopped and thegeneration and the emission of image light in the image-light generatingunit are stopped. Therefore, the display of the virtual image VIdisappears from the visual field VR of the user. As a result, the usercan easily visually recognize an external scene.

The detecting unit 147 may request the image processing unit 160 tochange the image data Data to dummy data of a single color black.Consequently, images rendered on the LCDs (the right. LCD 241 and theleft LCD 242) are adjusted to dummy images of the single color black.The image-light generating unit emits image light corresponding to thedummy images. Therefore, the virtual image VI displayed in the visualfield VR of the user is displayed as if the virtual image VI disappears.Therefore, the user can easily visually recognize an external scene.

As explained above, according to the third embodiment, the detectingunit 147 detects, using angular velocity (change information indicatinga change in a direction with respect to the initial position) detectedby the gyro sensor 74, the movement of the head exceeding the fixedamount of the user wearing the image display unit 20 b. When it isassumed that the user shifts the user's attention from the virtualimage, specifically, when the movement of the head exceeding the fixedamount is detected, to reduce the visibility of the virtual image VI,the control unit 10 b adjusts the luminance of the light sources (theright backlight 221 and the left backlight 222) to adjust the luminanceof the image-light generating unit or adjusts images generated by thedisplay devices (the right LCD 241 and the left LCD 242) to adjust imagelight generated by the image-light generating unit. As a result, in thehead mounted display HMb, it is possible to detect the movement of thehead exceeding the fixed amount of the user wearing the image displayunit 20 b and improve the visibility of an external scene.

D. Fourth Embodiment

In a fourth embodiment of the invention, a configuration for furtherperforming, in addition to the movement detection processing,line-of-sight detection processing for detecting the direction of a lineof sight of a user wearing an image display unit and erasing a virtualimage is explained. In the following explanation, only components havingconfigurations and operations different from those in the thirdembodiment are explained. In the figures, components same as those inthe third embodiment are denoted by reference numerals and signs same asthose in the third embodiment explained above and detailed explanationof the components is omitted.

D-1. Configuration of a Head-Mounted Display Device

FIG. 18 is a functional block diagram showing the configuration of ahead mounted display HMc in the fourth embodiment. The head mounteddisplay HMc is different from the head mounted display HMb in the thirdembodiment shown in FIG. 13 in that the head mounted display HMcincludes a control unit 10 c instead of the control unit 10 b andincludes an image display unit 20 c instead of the image display unit 20b.

The image display unit 20 c further includes an eye camera 73functioning as an image pickup unit. The eye camera 73 in thisembodiment is arranged on the inner surface of the right optical panel26 (i.e., a surface on a side opposed to the eye of the user in a wornstate of the head mounted display HMc). The eye camera 73 in thisembodiment includes an infrared light and a CCD camera and acquires aneyeball image of the right eye of the user.

The control unit 10 c includes threshold information CIIa instead of thethreshold information CII and includes a detecting unit 147 a instead ofthe detecting unit 147. In the threshold information CIIa, a thirdthreshold is further stored in advance in addition to the first andsecond thresholds. The third threshold is used in the line-of-sightdetection processing (explained later). The detecting unit 147 aexecutes the line-of-sight detection processing in parallel to themovement detection processing (FIG. 15). The line-of-sight detectionprocessing is processing for detecting the direction of the line ofsight of the user wearing the image display unit 20 b and erasing avirtual image.

D-2. Movement Detection Processing

The movement detection processing for the head mounted display HMc inthe fourth embodiment is the same as the movement detection processingin the third embodiment shown in FIG. 15.

D-3. Line-of-Sight Detection Processing

FIG. 19 is a flowchart for explaining a procedure of the line-of-sightdetection processing for the head mounted display HMc in the fourthembodiment. First, the detecting unit 147 a determines whether thesetting button 72 provided in the image display section 20 c is pressed(step S402). Details are the same as step S402 of the movement detectionprocessing (FIG. 15). When the setting button 72 is pressed (YES in stepS402), the detecting unit 147 a reads the third threshold from thethreshold information CIIa (step S502). After acquiring the threshold,the detecting unit 147 a acquires an eyeball image picked up by the eyecamera 73 (step S504). The detecting unit 147 a analyzes the acquiredeyeball image (step S506).

FIGS. 20A to 20C are explanatory diagrams concerning a method ofanalyzing an eyeball image. FIG. 20A shows a side general view of theeye of the user wearing the head mounted display HMc. The detecting unit147 a analyzes an eyeball image picked up by the eye camera 73 tothereby calculate a line-of-sight movement amount indicating how much aniris AE of the user moves with respect to a center position AX of theeye. In this embodiment, the line-of-sight movement amount isrepresented by a combination of a movement amount in the x axisdirection and a movement amount in the y axis direction. FIG. 20B is adiagram of a case in which the iris AE is located in the center positionAX and shows a line-of-sight movement amount (x axis, y axis) in thecase. In this example, since the iris AE is present on the centerposition AX, the line-of-sight movement amount is (0, 0). FIG. 20C is adiagram of a case in which the iris AE of the user moves to the leftside with respect to the center position AX and shows a line-of-sightmovement amount (x axis, y axis) in the case. In this example, since theiris AE moves from the center position AX in the horizontal direction,the line-of-sight movement amount is (+10, 0).

In step S508 in FIG. 19, the detecting unit 147 a determines whether atleast one of the movement amount on the x axis and the movement amounton the y axis of the iris calculated in step S506 exceeds a thirdthreshold Th3 (step S508). When both the movement amounts on the x axisand the y axis of the iris are equal to or smaller than the thirdthreshold Th3 (NO in step S508), the detecting unit 147 a shifts theprocessing to step S504 and acquires an eyeball image of the eye camera73 again. On the other hand, when at least one of the movements amountson the x axis and the y axis of the iris exceeds the third threshold Th3(YES in step S508), the detecting unit 147 a turns off the backlights(step S414). Details are the same as step S414 of the movement detectionprocessing (FIG. 15).

After turning off the backlights, the detecting unit 147 a acquires aneyeball image of the eye camera 73 (step S510). The detecting unit 147 aanalyzes the acquired eyeball image and calculates a line-of-sightmovement amount of the iris (step S512). Details are the same as stepS506. The detecting unit 147 a determines whether at least one of acalculated movement amount on the x axis of the iris and a calculatedmovement amount on the y axis of the iris is equal to or larger than thethird threshold Th3 (step S514). When both the movement amounts on the xaxis and the y axis are smaller than the third threshold Th3 (NO in stepS514), the detecting unit 147 a shifts the processing to step S510 andacquires an eyeball image of the eye camera 73 again. On the other hand,when at least one of the movement amounts on the x axis and the y axisis equal to or larger than the third threshold Th3 (YES in step S514),the detecting unit 147 a turns on the backlights. Details are the sameas step S420 of the movement detection processing (FIG. 15).

As explained above, according to the fourth embodiment, the detectingunit 147 a analyzes an eyeball image of the user picked up by the eyecamera 73 to acquire a line-of-sight movement amount including amovement amount in the x axis direction and a movement amount in the yaxis direction with respect to the center position AX of the iris AE ofthe user. When the line-of-sight movement amount exceeds the thirdthreshold Th3 set in advance, the control unit 10 b adjusts theluminance of the image-light generating unit or adjusts image lightgenerated by the image-light generating unit to reduce the visibility ofa virtual image. As a result, according to this embodiment, in additionto the effect of the third embodiment, it is possible to detect that theuser shifts the line of sight away from the center position and improvethe visibility of an external scene.

E. Modifications

The invention is not limited to the embodiments explained above. Variousconfigurations can be adopted without departing from the spirit of theinvention. For example, the functions realized by software may berealized by hardware. Besides, modifications explained below arepossible.

E1. Modification 1

In the embodiments, the configuration of the head mounted display isexemplified. However, the configuration of the head mounted display canbe arbitrarily set without departing from the spirit of the invention.For example, addition, deletion, conversion, and the like of thecomponents are possible.

In the embodiments, for convenience of explanation, the control unitincludes the transmitting units (51 and 52) and the image display unitincludes the receiving units (53 and 54). However, both the transmittingunits (51 and 52) and the receiving units (53 and 54) in the embodimentshave a function of being capable of performing bidirectionalcommunication and can function as transmitting and receiving units.

For example, as shown in FIG. 21, the connecting unit may be omitted andthe control unit and the image display unit may be capable ofcommunicating with each other by radio. Specifically, the control unitfurther includes a radio communication unit (81) and further includes aradio communication unit (82) and a power supply (280) in the imagedisplay unit. In this case, the radio communication unit 81 functions asthe transmitting units (51 and 52) in the embodiments and the radiocommunication unit 82 functions as the receiving units (53 and 54) inthe embodiments.

For example, the configurations of the control unit and the imagedisplay unit shown in FIG. 1 can be arbitrarily changed. Specifically,for example, the touch panel may be omitted from the control unit andmay be operated only with the cross key. Another interface for operationsuch as a stick for operation may be provided in the control unit.Devices such as a keyboard and a mouse may be able to be connected tothe control unit to receive inputs from the keyboards and the mouse. Acommunication unit employing Wi-Fi (wireless fidelity) or the like maybe provided in the control unit.

For example, the control unit shown in FIG. 1 is connected to the imagedisplay unit via a wired signal transmission line. However, the controlunit and the image display unit may be connected via a wireless signaltransmission line such as a wireless LAN, infrared communication, orBluetooth (registered trademark).

For example, the head mounted display is a transmissive head mounteddisplay of a binocular type. However, the head mounted display may be anontransmissive head mounted display in which an external scene isblocked in a state in which the user wears the head mounted display. Thehead mounted display may be a head mounted display of a monocular type.

For example, the image-light generating unit is configured using theleft and right-backlight control units, the left and right-LCD controlunits, the left and right backlights, and the left and right LCDs.However, instead of these devices, an organic EL (OrganicElectro-Luminescence) and an organic-EL control unit may be used. Inthat case, the organic EL and the organic-EL control unit correspond to“image-light generating unit”.

For example, the functional units such as the detecting unit, the imageprocessing unit, the display control unit, and the sound processing unitare described as being realized by the CPU expanding the computerprogram stored in the ROM or the hard disk on the RAM and executing thecomputer program. However, these functional units may be configuredusing an ASIC (Application Specific Integrated Circuit) designed torealize the functions.

For example, in the embodiments, the head mounted display is the headmounted display, the image display unit of which is worn likeeyeglasses. However, the image display unit may be a usual flat displaydevice (a liquid crystal display device, a plasma display device, anorganic EL display device, etc.). Even in this case, the connectionbetween the control unit and the image display unit may be connectionvia a wired signal transmission line or may be connection via a wirelesssignal transmission line. Consequently, it is possible to use thecontrol unit as a remote controller of the usual flat display device.

As the image display unit, an image display unit having another shapesuch as an image display unit worn like a cap may be adopted instead ofthe image display unit worn like eyeglasses. As the earphones, earphonesof an ear hooking type or a headband type may be adopted. The earphonesmay be omitted.

For example, in the embodiments, the secondary battery is used as thepower supply. However, the power supply is not limited to the secondarybattery. Various batteries can be used as the power supply. For example,a primary battery, a fuel battery, a solar battery, and a thermalbattery may be used.

E2. Modification 2

In the embodiments, the image processing unit outputs the same imagedata as image data for the right eye and image data for the left eye.However, the image processing unit may be configured to be capable ofoutputting different image data as image data for the right eye andimage data for the left eye to cause the user to visually recognize a 3Dvirtual image.

E3. Modification 3

In the user-side see-through processing (FIGS. 5 and 11) in theembodiments, the direction determining unit determines whether thecontrol unit and the image display unit are opposed to each otheraccording to whether the direction determining unit receives outputsignals from the head-side light receiving unit and thecontrol-unit-side light receiving unit. However, the form in theembodiments is only an example. The direction determining unit candetermine whether the control unit and the image display unit areopposed to each other using various methods.

For example, one of the set of the head-side light emitting unit and thecontrol-unit-side light receiving unit or the set of thecontrol-unit-side light emitting unit and the head-side light receivingunit may be omitted. Consequently, as in the embodiments, in the headmounted display, it is possible to detect, with a configurationdifferent from that in the past, that the user attempts to operate thecontrol unit. It is possible to improve the visibility of an externalscene in the image display unit. Since only one set of the lightemitting unit and the light receiving unit is necessary, theconfiguration can be realized at low cost.

For example, instead of the light emitting units (the head-side lightemitting unit and the control-unit-side light emitting unit) and thelight receiving units (the head-side light emitting unit and thecontrol-unit-side light emitting unit) or in addition to the lightemitting units and the light receiving units, the head mounted displaymay includes, in the image display unit, an acceleration-informationdetecting unit for detecting acceleration information of the imagedisplay unit and an angular-velocity-information detecting unit fordetecting angular velocity information of the image display unit. As theacceleration-information detecting unit, for example, an accelerationsensor can be used. As the angular-velocity-information detecting unit,for example, a gyro sensor can be used.

When the head mounted display includes the acceleration-informationdetecting unit and the angular-velocity-information detecting unitinstead of the light emitting units and the light receiving units, thedirection determining unit can determine that the operation surface andthe image display unit are opposed to each other (YES in step S104) whenthe movement of the image display unit derived from the accelerationinformation and the angular velocity information (i.e., the movement ofthe head of the user wearing the image display unit) coincides withmovement assumed in advance (e.g., the user moves the head downward tolook at the operation surface). Consequently, when the head mounteddisplay includes the acceleration-information detecting unit and theangular-velocity-information detecting unit instead of the lightemitting units and the light receiving units, it is possible to obtaineffects same as those of the embodiments.

When the head mounted display includes the acceleration-informationdetecting unit and the angular-velocity-information detecting unit inaddition to the light emitting units and the light receiving units, thedirection determining unit can further determine whether the movement ofthe image display unit (the movement of the head of the user) derivedfrom the acceleration information and the angular velocity informationcoincides with movement assumed in advance (e.g., downward movement ofthe head) when the direction determining unit receives output signalsfrom both of the head-side light receiving unit and thecontrol-unit-side light receiving unit (YES in step S104). Consequently,it is possible to further improve accuracy of the determinationconcerning “whether the user faces the operation surface” in theuser-side see-through processing.

When the head mounted display includes the acceleration-informationdetecting unit and the angular-velocity-information detecting unit inaddition to the light emitting units and the light receiving units andthe backlights include plural light emitting members (LEDs), thedirection determining unit can further determine, according to themovement of the image display unit (the movement of the head of theuser) derived from the acceleration information and the angular velocityinformation, which of the plural light emitting members are turned offwhen the backlights are turned of f (step S106). Consequently, since thedisplay of the virtual image in a portion equivalent to a regioncorresponding to the movement of the head of the user is erased, it ispossible to improve convenience for the user.

The head mounted display may include a distance measurement sensor inthe image display unit instead of, or in addition to, the light emittingunits and the light receiving units. When the head mounted displayincludes the distance measurement sensor instead of the light emittingunits and the light receiving units, it is possible to obtain effectssame as those in the embodiments. When the head mounted display includesthe distance measurement sensor in addition to the light emitting unitsand the light receiving units, it is possible to further improveaccuracy of the determination concerning “whether the user faces theoperation surface” in the user-side see-through processing. Further, thedirection determining unit may adjust the luminance of the light sourceaccording to a distance between the control unit and the image displayunit derived from a measurement result of the distance measurementsensor. For example, when the distance between the control unit and theimage display unit is small, the direction determining unit can reducethe luminance of the light source (i.e., reduce the luminance of theimage-light generating unit and make the display of the virtual image VIpale). When the distance between the control unit and the image displayunit is large, the direction determining unit can increase the luminanceof the light source (i.e., increase the luminance of the image-lightgenerating unit and make the display of the virtual image VI dark).

When the head mounted display includes the distance measurement sensorin the image display unit instead of the light emitting units and thelight receiving units or in addition to the light emitting units and thelight receiving units and the backlights include plural light emittingmembers (LEDs), in turning off the backlights (step S106), the directiondetermining unit can determine, according to a distance between thecontrol unit and the image display unit derived from a measurementresult of the distance measurement sensor, which of the plural lightemitting members are turned off. For example, when the distance betweenthe control unit and the image display unit is small, the directiondetermining unit can increase the number of light emitting members to beturned off (i.e., increase the area of a region without image display inthe virtual image VI). When the distance between the control unit andthe image display unit is large, the direction determining unit canreduce the number of light emitting members to be turned off (i.e.,reduce the area of the region without image display in the virtual imageVI).

An image pickup unit (e.g., a CCD camera) may be provided in the controlunit or the image display unit instead of the light emitting units andthe light receiving units or in addition to the light emitting units andthe light receiving units. When the image pickup unit is provided in thecontrol unit, the direction determining unit analyzes a picked-up imageand determines whether the optical panel housing of the image displayunit is included in the image. When the optical panel housing isincluded in the picked-up image, the direction determining unit candetermine that the operation surface and the image display unit areopposed to each other (YES in step S104). On the other hand, when theimage pickup unit is provided in the image display unit, the directiondetermining unit analyzes a picked-up image and determines whether thetouch pad and the cross key arranged on the operation surface of thecontrol unit are included in the image. When the touch pad and the crosskey are included in the picked-up image, the direction determining unitcan determine that the operation surface and the image display unit areopposed to each other (YES in step S104). When the image pickup unit isprovided instead of the light emitting units and the light receivingunits, it is possible to obtain effects same as those of theembodiments. When the image pickup unit is provided in addition to thelight emitting units and the light receiving units, it is possible tofurther improve accuracy of the determination concerning “whether theuser faces the operation surface” in the user-side see-throughprocessing.

E4. Modification 4

In the user-side see-through processing (FIG. 11) in the secondembodiment, when contact is detected by the contact detecting unit, thedirection determining unit drives the light emitting units (thehead-side light emitting unit and the control-unit-side light emittingunit). However, the form in the embodiments is only an example. Variousmodifications are possible.

For example, the control unit may include, instead of the contactdetecting unit, an acceleration-information detecting unit (anacceleration sensor) for detecting acceleration information of thecontrol unit. In this case, when the acceleration-information detectingunit detects the tilt of the control unit, the direction determiningunit can determine YES in step S202. Consequently, when the control unitincludes the acceleration-information detecting unit instead of thecontact detecting unit, it is possible to obtain effects same as thosein the embodiments.

For example, the control unit can use the touch pad as the contactdetecting unit. Consequently, it is possible to realize the head mounteddisplay at low cost.

E5. Modification 5

In the movement detection processing (FIGS. 14A and 14B) in theembodiments, the detecting unit detects, according to an output valuefrom the gyro sensor, the movement of the head exceeding the fixedamount of the user. However, the form in the embodiments is only anexample. The detecting unit can detect the movement of the headexceeding the fixed amount of the user using various methods.

For example, in step S410 of the movement detection processing, thedetecting unit determines whether at least one of the angle of the xaxis and the angle of the y axis obtained from the output value of thegyro sensor exceeds the same threshold (the first threshold Th1).However, in general, the movement of the face in the horizontaldirection is larger than the movement of the head in the perpendiculardirection (the vertical direction). Therefore, two kinds of the firstthreshold Th1 may be prepared to be associated with the perpendiculardirection (for the y axis) and horizontal direction (for the x axis).The same holds true concerning step S412.

For example, in the embodiments, the two-axis angular velocity sensor ofthe piezoelectric vibration type is used as the gyro sensor to detectthe movement of the head in the perpendicular direction (the verticaldirection) and the movement of the face in the horizontal direction ofthe user. However, as the gyro sensor, angular velocity sensors ofvarious types such as one-axis and three-axis types can be used. If thenumber of axes that the gyro sensor can detect increases, it is possibleto finely detect the movement of the head of the user.

For example, an acceleration sensor may be further provided in the imagedisplay unit and, in addition to the determination in steps S406 to S412of the movement detection processing, the detecting unit may determine“the movement of the head exceeding a fixed amount” using the tilt, themovement, the vibration, and the impact of the image display unitobtained from an output value of the acceleration sensor. If both of thegyro sensor and the acceleration sensor are used, it is possible toimprove accuracy of the determination in the detecting unit. In thiscase, the acceleration sensor corresponds to “acceleration detectingunit” in the appended claims.

For example, the image display unit may include, instead of the gyrosensor, an electronic compass that detects, using terrestrial magnetism,a direction in which the image display unit faces and an accelerationsensor that detects the acceleration of the image display unit. In thiscase, the determining unit can perform the determination in step S410 ofthe movement detection processing using an angle calculated from thedirection detected by the electronic compass and can perform thedetermination in step S412 of the movement detection processing usingthe acceleration detected by the acceleration sensor. The electroniccompass corresponds to “terrestrial-magnetism detecting unit” in theappended claims. The acceleration sensor corresponds to “accelerationdetecting unit” in the appended claims. Consequently, in theconfiguration including the electronic compass and the accelerationsensor instead of the gyro sensor, it is possible to obtain effects sameas those of the embodiments.

E6. Modification 6

In the movement detection processing (FIGS. 14A and 14B) in theembodiments, the detecting unit is triggered by pressing of the settingbutton to set an initial position and continues the movement detectionprocessing. However, various changes of the trigger are possible.

For example, step S402 of the movement detection processing may beomitted. The detecting unit may set an initial position and execute stepS404 and subsequent steps when the head mounted display HMb is poweredon.

For example, in step S402 of the movement detection processing, when thedetecting unit determines that the user keeps the same posture for afixed time, the detecting unit may set an initial position and executestep S404 and subsequent steps. Specifically, the detecting unit candetermine the posture of the user by monitoring changes in an angle andangular velocity obtained from an output value of the gyro sensor.Consequently, since the operation of the setting button by the user isunnecessary, it is possible to improve convenience of use (usability) ofthe head mounted display HMb.

For example, in step S402 of the movement detection processing, when thedetecting unit detects startup of a specific application (e.g., movementreproduction software) installed in the head mounted display HMb, thedetecting unit may set an initial position and execute steps S404 andsubsequent steps. The specific application can be arbitrarily set andcan be designated by the user.

For example, when an acceleration sensor is further provided in theimage display unit, in step S402 of the movement detection processing,the detecting unit may detect “a movement of patting the image displayunit” and set an initial position on the basis of acceleration obtainedfrom an output value of the acceleration sensor and execute step S404and subsequent steps. Consequently, since the setting button is omitted,it is possible to improve convenience of use (usability) of the headmounted display HMb.

E7. Modification 7

In the embodiments, the arrangement of the acquiring unit for acquiringchange information is exemplified. However, the arrangement of theacquiring unit for acquiring change information is only an example.Various changes of the arrangement are possible.

For example, in the third embodiment, the gyro sensor functioning as theacquiring unit is arranged on the inside of the housing of the rightdisplay unit. However, gyro sensors may be arranged in both the rightdisplay unit and the left display unit. In that case, in the movementdetection processing (FIGS. 14A and 14B), output values of the left andright gyro sensors may be compared to calculate an angle and the like.

For example, in the fourth embodiment, the eye camera functioning as theacquiring unit is arranged on the inner surface of the right opticalpanel. However, eye cameras may be arranged on both the inner surface ofthe right optical panel and the inner surface of the left optical panel.In that case, in the line-of-sight detection processing (FIG. 19), aline-of-sight movement amount may be calculated using the movement ofthe left and right irises detected by the left and right eye cameras.

The entire disclosure of Japanese Patent Application Nos. 2011-066373,filed Mar. 24, 2011 and 2011-066393, filed Mar. 24, 2011 are expresslyincorporated by reference herein.

What is claimed is:
 1. A head-mounted display device comprising: animage display unit including an image-light generating unit thatgenerates image light representing an image and emits the image lightand a light guide unit that guides the emitted image light to an eye ofa user, the image display unit causing the user to visually recognize avirtual image; and a control unit that is connected to the image displayunit and controls image display by the image display unit, the controlunit including an operation surface that controls an operation of thehead-mounted display device based on finger operation by the userdetected on the operation surface; and a direction determining unit thatdetermines whether the user faces the operation surface of the controlunit, wherein if the direction determination unit determines that theuser faces the operation surface, the control unit adjusts luminance ofthe image-light generating unit or adjusts the image light generated bythe image-light generating unit to reduce visibility of the virtualimage.
 2. The head-mounted display device according to claim 1, theimage display unit further including a distance measurement sensor thatmeasures a distance between the control unit and the image display unit.3. The head-mounted display device according to claim 2, wherein thedirection determining unit adjusts the luminance of the image-lightgenerating unit based on the measured distance between the control unitand the image display unit.
 4. The head-mounted display device accordingto claim 2, wherein the direction determining unit adjusts the imagelight generated by the image-light generating unit based on the measureddistance between the control unit and the image display unit.
 5. Thehead-mounted display device according to claim 1, further comprising: animage pickup unit, wherein the direction determining unit determineswhether the operation surface and the image display unit are opposed toeach other based on analysis of a picked-up image obtained from theimage pickup unit.
 6. The head-mounted display device according to claim5, wherein the image pickup unit is provided in the control unit, andthe analysis of the picked-up image includes determining whether aportion of the image display unit is included in the picked-up image. 7.The head-mounted display device according to claim 5, wherein the imagepickup unit is provided in the image display unit, and the analysis ofthe picked-up image includes determining whether a portion of thecontrol unit is included in the picked-up image.
 8. A control method fora head-mounted display device including an operation surface, thecontrol method comprising: (a) causing a user to visually recognize avirtual image by using an image-light generating unit that generatesimage light representing an image and emits the image light and a lightguide unit that guides the emitted image light to an eye of the user;(b) controlling image display in (a) by using the operation surface tocontrol an operation of the head-mounted display device based on fingeroperation by the user detected on the operation surface; (c)determining, by a direction determination unit, whether the user facesthe operation surface to control the operation of the head-mounteddisplay device; and (d) if the direction determination unit determinesthat the user faces the operation surface, adjusting luminance of theimage-light generating unit or adjusting the image light generated bythe image-light generating unit to reduce visibility of the virtualimage.
 9. The control method according to claim 8, further comprising:measuring a distance between the control unit and the image displayunit.
 10. The control method according to claim 9, further comprising:adjusting the luminance of the image-light generating unit based on themeasured distance between the control unit and the image display unit.11. The control method according to claim 9, further comprising:adjusting the image light generated by the image-light generating unitbased on the measured distance between the control unit and the imagedisplay unit.
 12. The control method according to claim 8, furthercomprising: determining whether the operation surface and the imagedisplay unit are opposed to each other based on analysis of a picked-upimage obtained from an image pickup unit.
 13. The control methodaccording to claim 12, wherein when the image pickup unit is provided inthe control unit, the analysis of the picked-up image includesdetermining whether a portion of the image display unit is included inthe picked-up image.
 14. The control method according to claim 12,wherein when the image pickup unit is provided in the image displayunit, the analysis of the picked-up image includes determining whether aportion of the control unit is included in the picked-up image.